Columbia Kermit/kermit.zip/newsgroups/misc.20010306-20010921/[e-mail] _Fri Mar 16 15:21:31 2001.msg

Internet Message Format | 2020-01-01 | 4KB

Return-Path: <[e-mail]>
Received: from newsmaster.cc.columbia.edu (newsmaster.cc.columbia.edu [128.59.59.30])
    by monire.cc.columbia.edu (8.9.3/8.9.3) with ESMTP id PAA15753
    for <[e-mail]>; Fri, 16 Mar 2001 15:21:30 -0500 (EST)
Received: (from news@localhost)
    by newsmaster.cc.columbia.edu (8.9.3/8.9.3) id PAA24252
    for [e-mail]; Fri, 16 Mar 2001 15:19:41 -0500 (EST)
X-Authentication-Warning: newsmaster.cc.columbia.edu: news set sender to <news> using -f
From: [e-mail] (Frank da Cruz)
Subject: Re: Confirm FTP Upload
Date: 16 Mar 2001 20:19:39 GMT
Organization: Columbia University
Message-ID: <98tsgr$nlp$[e-mail]>
To: [e-mail]

: Rick wrote:
: >
: > My company is currently processing two large text files on an E10K box and
: > then uploading them to an anonymous FTP server. One file is processed
: > daily and the other weekly, the processing and upload is via crontab. The
: > customer that receives this text file has now requested that a separate
: > "control" file be put on the FTP site which they can look for and if the
: > control file is available they will continue to retrieve the text
: > file. The intent is that this control file will only be there if the file
: > that I sent was received by the FTP site without error. I'm a little
: > confused on how to verify that what I sent was received...
:
This is an insolable problem.  However, if you transferred the file in the
right mode (text versus binary) and there were no errors, the chances are
vanishingly small that file was not transferred correctly, since TCP and IP
both provide error detection and correction.  If you transfer in binary mode,
and the size the is the same, that's another good indicator of succees.

Anyway, I think what the customer really wants is to know that an upload
has finished before grabbing the file from the upload area, which is an
easier problem to solve.

: > ... the best thing I
: > can think of is that I run another process that goes out to the FTP site
: > downloads the file, compares it character by character and if it matches
: > then send the control file.
:
This would not catch some hypothetical errors that are both systematic and
reversible.

: > This doesn't seem like a very good method
: > because I'm checking to see if one operation has failed with an operation
: > that could fail.
:
Right.  First I'd suggest you read the following FTP scripting tutorial,
which discusses the same problem you're trying to address:

  http://www.columbia.edu/kermit/ftpscript.html


The tutorial pertains to the new Kermit Project FTP client, is designed to
handle such situations.  Example 5 applies to your query.

If you want an even greater level of confidence, you might consider using
Kermit protocol rather than FTP because:

 . FTP protocol includes no error detection or correction whatsoever;
   it relies completely on the underlying transport for that.  Kermit,
   on the other hand, applies rigorous checking on top of TCP and IP.

 . FTP protocol includes no mechanism for confirming that a file was
   was fully received.  The sender simply closes the connection at the
   end of the file.  Kermit protocol, on the other hand, sends an "end
   of file" packet at the end of each file and requires positive
   confirmation from the receiver before declaring the transfer
   successfully terminated.

Internet Kermit servers are available that can be used just like Internet
FTP servers, and that also include security and privacy methods lacking
from most FTP servers.  For more information see:

  http://www.columbia.edu/kermit/cuiksd.html

Also see the following case study:

  http://www.columbia.edu/kermit/case10.html

which discusses "atomic file movement" in detail, which is the topic of
your query.

- Frank